An embedded randomised controlled retention trial of personalised text messages compared to non(cid:406)personalised text messages in an orthopaedic setting (cid:446)version (cid:696)(cid:402) peer review(cid:403) (cid:696)

Several studies have investigated whether personalising trial Background(cid:403) documentation can aid recruitment and retention(cid:404) We did a (cid:415)study within a trial(cid:416) (cid:442)SWAT(cid:443) evaluating the effectiveness of a personalised text message compared to a non(cid:406)personalised text message(cid:401) on the retention rate in a large orthopaedic trial(cid:404) The SWAT was embedded in the Knee Replacement Bandaging Methods(cid:403) Study (cid:442)KReBS(cid:443) trial(cid:404) The primary outcome was the proportion of (cid:696)(cid:697)(cid:406)month a of personalized vs(cid:404) reminder in a randomized orthopaedic primary outcome questionnaire return secondary outcomes proportion of completed questionnaires(cid:401) and to questionnaire return(cid:404) There was no difference in any outcome between groups(cid:404)


Abstract
Several studies have investigated whether personalising trial Background documentation can aid recruitment and retention We did a study within a trial SWAT evaluating the effectiveness of a personalised text message compared to a non personalised text message on the retention rate in a large orthopaedic trial The SWAT was embedded in the Knee Replacement Bandaging Methods Study KReBS trial The primary outcome was the proportion of month questionnaires returned Secondary outcomes were the proportion of questionnaires completed and time to questionnaire return Binary data were analysed using logistic regression and time to return using Cox proportional hazards regression Odds ratios OR and hazard ratios HR are presented with associated confidence intervals CI and p values In total participants were included in the SWAT In the Results personalised group of participants returned a questionnaire compared to in the non personalised group The absolute difference in return rate was CI to p There was no evidence of a difference between the groups in the likelihood of returning a questionnaire OR

Introduction
Clinical trialists have identified the recruitment and retention of participants as key issues for randomised controlled trials (RCT) 1,2 .
Several studies have investigated whether personalising trial documentation can aid recruitment and retention 3,4 . Recently, Cochrane et al. looked at the effect of personalised text messages compared to standard text messages in improving retention rates 5 . This study was carried out in response to a number of embedded trials evaluating the effectiveness of SMS messages in improving retention rates 6-11 , alongside a study suggesting personalised messages increased the payment of delinquent fines 12 .
To further add to the evidence on the effectiveness of personalised text messages, we did a 'study within a trial' (SWAT) evaluating the effectiveness of a personalised text message compared to a non-personalised text message on postal questionnaire response rates in a large orthopaedic trial.

Design
This paper details the methods and results of a SWAT embedded within the prospectively registered Knee Replacement Bandaging Study (KReBS) RCT (ISRCTN87127065, registered on 20 February 2017). KReBS evaluated the effectiveness of a two-layer compression bandage compared with a standard wool and crepe bandage applied post-operatively on patient-reported outcomes in total knee replacement patients 13 .

Participants
The SWAT was conducted in 26 NHS hospital trust sites and was implemented at the start of the study. All KReBS participants were eligible for this SWAT provided they had opted in to receiving SMS messages and were not deceased or withdrawn from follow-up before being due to be sent their 12-month postal questionnaire.

Intervention
Participants in the SWAT were sent either a personalised or non-personalised text message (Table 1) four days after their 12-month questionnaire was sent.

Outcomes
The primary outcome was the proportion of participants who returned a 12-month questionnaire. Secondary outcomes were the proportion of participants who completed the questionnaire and time to questionnaire return. A questionnaire was considered complete if the participant had answered 11 or more questions of the 12-item host trial primary outcome, the Oxford Knee Score 14 .

Sample size
Since this was an embedded trial, the sample size was determined by the number of participants in the main KReBS trial 13 , which aimed to recruit 2600 participants.

Randomisation
Participants were randomised into the SWAT using simple randomisation in a 1:1 allocation ratio. The allocation schedule was generated by a researcher at the York Trials Unit not involved in the recruitment or follow-up of participants.

Blinding
Participants were not informed of their explicit participation in the SWAT, but due to the nature of the intervention could not be blinded to whether the text was personalised or non-personalised. Similarly, it was not possible to blind research staff to SWAT allocation.

Approvals
The SWAT was approved by the Research Ethics Committee North East -Newcastle and North Tyneside on 13/04/2018 (REC Number 16/NE/0400; Amendment Number 16/NE/0400/AM14). As the SWAT was deemed to be low risk, explicit informed consent was not obtained for participation.

Statistical analysis
Analyses were carried out using Stata v16.0 15 . A diagram detailing the flow of participants through the SWAT is provided, and baseline characteristics are presented by SWAT allocation. Outcomes are summarised descriptively. Statistical tests were two-sided using a 5% significance level, and were done on an intention to treat basis. All analyses (except the calculation of the absolute difference in return rate which was estimated using the two-sample test of proportions) used mixed The proportion of participants who returned a 12-month questionnaire, and proportion complete, was analysed using logistic regression. A second SWAT evaluating receipt of a pen on response rates was also embedded in KReBS at 12 months 16 .
In a sensitivity analysis, we additionally adjusted the primary model for pen SWAT allocation.
Time to questionnaire return was analysed using a Cox proportional hazards shared frailty model. Participants who did not return a questionnaire were censored at 90 days.

Results
In total, 2335 participants were recruited into the KReBS trial and 1470 were randomised to the SWAT (Figure 1). The average age was 66.8 years and 54.0% were female (Table 2 17 ). Five participants died or withdrew following randomisation and as a result 723 participants in the personalised group, and 742 in the non-personalised group, were sent a 12-month questionnaire and were included in the analysis. Of these, 680 (94.1%) of the 723 participants in the personalised group, and 701 (94.5%) of the 742 in the non-personalised group, were sent a text.
In the personalised group, 644/723 (89.1%) participants returned a questionnaire, compared to 654/742 (88.1%) in the non-personalised group (Table 3 17 ). The absolute difference in return rate was 0.9% (95% CI: -2.3% to 4.2%; p=0.57). There was no evidence of a difference between the groups in the likelihood of returning a questionnaire (OR 1.09; 95% CI: 0.79 to 1.51; p=0.61), the likelihood of returning a complete questionnaire (OR 1.11; 95% CI: 0.82 to 1.51; p=0.50) nor in time to return (HR 1.05; 95% CI: 0.94 to 1.17; p=0.40). In total, 1465 participants were also randomised to the pen SWAT. When the primary model was repeated with the addition of pen SWAT allocation, the results remained the same.

Discussion
This embedded trial found little evidence to suggest personalised text messages are more effective than non-personalised text messages in encouraging return and completion of questionnaires. The trial did not find evidence of a statistically significant difference between groups in any of the outcomes, although effect size estimates favoured the personalised group. On the other hand, while Cochrane and colleagues also did not find evidence of a statistically significant difference between groups, estimates of effect mostly favoured the non-personalised group 5 .
The SWAT had a large sample size, which means the results can be generalised to other orthopaedic studies. However, completion rate was calculated as a proportion of all SWAT participants rather than all SWAT participants who returned a questionnaire, and as a result questionnaire completion was highly correlated with questionnaire return. In addition, some participants included in the analysis did not receive a text message.

Conclusion
This SWAT adds to the growing evidence base for whether personalised trial documentation, in particular text messages, are effective.

Data availability
Underlying data Open Science Framework: Underlying data and CONSORT diagram for an embedded randomised controlled retention trial of personalised text messages compared to non-personalised

If applicable is the statistical analysis and its interpretation appropriate Yes
Are all the source data underlying the results available to ensure full reproducibility Yes

Are the conclusions drawn adequately supported by the results Yes
No competing interests were disclosed Competing Interests Reviewer Expertise Research methodology trial design and reporting

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard
The benefits of publishing with F Research Your article is published within days with no editorial bias You can publish traditional articles null negative results case reports data notes and more The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre submission enquiries contact research f com